Prior results have shown that LLMs released before 2024 can't leverage 'filler tokens'—unrelated tokens prior to the model's final answer—to perform additional computation and improve performance.[1]I did an investigation on more recent models (e.g. Opus 4.5) and found that many recent LLMs improve substantially on math problems when given filler tokens.That is, I force the LLM to answer some math question immediately without being able to reason using Chain-of-Thought (CoT), but do give the LLM filter tokens (e.g., text like "Filler: 1 2 3 ...") before it has to answer.Giving Opus 4.5 filler tokens[2] boosts no-CoT performance from 45% to 51% (p=4e-7) on a dataset of relatively easy (competition) math problems[3].I find a similar effect from repeating the problem statement many times, e.g. Opus 4.5's no-CoT performance is boosted from 45% to 51%.

Repeating the problem statement generally works better and is more reliable than filler tokens, especially for relatively weaker models I test (e.g. Qwen3 235B A22B), though the performance boost is often very similar, especially for Anthropic models.The first model that is measurably uplifted by repeats/filler is Opus 3, so this effect has been around for a [...]

---

Outline:

(03:59 ) Datasets

(06:01 ) Prompt

(06:58 ) Results

(07:01 ) Performance vs number of repeats/filler

(08:29 ) Alternative types of filler tokens

(09:03 ) Minimal comparison on many models

(10:05 ) Relative performance improvement vs default absolute performance

(13:59 ) Comparing filler vs repeat

(14:44 ) Future work

(15:53 ) Appendix: How helpful were AIs for this project?

(16:44 ) Appendix: more information about Easy-Comp-Math

(25:05 ) Appendix: tables with exact values

(25:11 ) Gen-Arithmetic - Repetitions

(25:36 ) Gen-Arithmetic - Filler

(25:59 ) Easy-Comp-Math - Repetitions

(26:21 ) Easy-Comp-Math - Filler

(26:45 ) Partial Sweep - Gen-Arithmetic - Repetitions

(27:07 ) Partial Sweep - Gen-Arithmetic - Filler

(27:27 ) Partial Sweep - Easy-Comp-Math - Repetitions

(27:48 ) Partial Sweep - Easy-Comp-Math - Filler

(28:09 ) Appendix: more plots

(28:14 ) Relative accuracy improvement plots

(29:05 ) Absolute accuracy improvement plots

(29:57 ) Filler vs repeat comparison plots

(30:02 ) Absolute

(30:30 ) Relative

(30:58 ) Appendix: significance matrices

(31:03 ) Easy-Comp-Math - Repetitions

(32:27 ) Easy-Comp-Math - Filler

(33:50 ) Gen-Arithmetic - Repetitions

(35:12 ) Gen-Arithmetic - Filler

The original text contained 9 footnotes which were omitted from this narration.

---

First published:
December 22nd, 2025

Source:
https://www.lesswrong.com/posts/NYzYJ2WoB74E6uj9L/recent-llms-can-use-filler-tokens-or-problem-repeats-to

---



Narrated by TYPE III AUDIO.

---